Boosting Weighted Linear Discriminant Analysis

We propose a novel approach to boosting weighted linear discriminant analysis (LDA) as a weak classifier. Combining Adaboost with LDA allows to select the most relevant features for classification at each boosting iteration, thus benefiting from feature correlation. The advantages of this approach include the use of a smaller number of weak learners to achieve a low error rate, improved classification performance due to the robustness and stable nature of LDA, and computational efficiency. The performance of the proposed method was evaluated on artificial data and additionally on two popular independent data sets: the Iris Data Set and the Breast Cancer Wisconsin Diagnostic Data Set, both publicly available at the University of California at Irvine Machine Learning Repository. Experimental results showed the superior accuracy of the proposed method over LDA and AdaBoost combined with other types of weak classifiers. The weighted LDA algorithm was proven to be equivalent to the traditional LDA in the case of uniform weight distributions.